The performance of an application update may be measured by running a controlled experiment. In the experiment, a control set of clients may execute a control version of the application and a variable set of clients may execute an alternate version of the application. During the testing of the different versions of the application, runtime measurement data may be cached from each of the clients and messages may be exchanged between the two sets of clients. Degradation and loss of data consistency in the stored data and messages may arise when attempting to deploy the updates to the two sets of clients.